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EXECUTIVE  SUMMARY 


Past  studies  conducted  by  the  National  Transportation  Safety  Board  (NTSB) 
and  the  Flight  Safety  Foundation  (FSF)  have  indicated  that  safety  has  not  been 
compromised  in  the  post-ATC  strike  environment.  This  report  summarizes  the 
results  of  a  study  sponsored  by  the  Office  of  Aviation  Safety  aimed  at  the 
quantitative  assessment  and  comparison  of  the  level  of  safety  of  the  ATC  sys¬ 
tem,  both  before  and  after  the  air  traffic  controllers'  strike. 

This  effort  focuses  on  "operational  errors"  as  a  surrogate  in  assessing 
the  safety  of  the  ATC  system.  This  is  not  to  say  that  other  parameters  and 
approaches  may  not  also  be  appropriate  in  this  regard;  however,  "operational 
errors"  was  chosen  since  it  directly  relates  to  controllers'  errors,  and  good 
pre-  and  post-strike  data  were  available. 

This  report  is  limited  to  the  treatment  of  operational  error  data.  The 
existence  of  qualifying  circumstances  and  considerations,  such  as  the  possible 
over-reporting  of  operational  errors  before  the  strike  and/or  the  under¬ 
reporting  of  such  errors  after  the  strike,  are  neither  accepted  nor  rejected. 
They  are  simply  not  dealt  with  in  this  report. 

The  frequency  of  operational  errors  and  the  volume  of  operations  at  FAA 
centers  and  towers  formed  the  statistical  basis  for  our  various  analyses. 
Operational  errors  are  recorded  on  a  daily  basis  and  were  summarized  for  our 
use  into  weekly  and  monthly  frequency  counts  along  with  those  corresponding 
numbers  of  operations. 

Our  first  level  of  study  simply  compared  the  absolute  numbers  of  opera¬ 
tional  errors  for  a  fixed  period  before  the  strike  data  of  August  3,  1981,  to 
a  similar  period  after  that  date.  The  results  of  that  comparison  follow: 


Terminal  Center  Total 

Operational  Errors  Operational  Errors  Errors 


Pre-Strike  Period 

(August  9,  1980,  to  278  171  449 

April  25,  1981) 


Post-Strike  Period  139  99  238 

(August  9,  1981,  to 
April  25,  1982) 


This  approach  in  itself  is  insufficient  for  firm  conclusions  since  it 
does  not  take  into  account  the  lower  level  of  operations  experienced  in  the 
post-strike  environment.  The  next  step  was  to  introduce  the  simple  concept 
of  a  rate.  This  was  done  by  comparing  the  number  of  errors  per  million 
operations  for  the  selected  pre-strike  period  to  a  similarly  selected  post¬ 
strike  period.  Those  results  are  as  follows: 


Terminal  Center 

Operational  Error  Rate  Operational  Error  Rate 
(Errors /Million  (Errors /Mill ion 

_ Operations)  _ Operations)  Total 


Pre-Strike  Period 

(August  9,  1980,  to  6.131  12.793  18.924 

April  25,  1981) 


Post-Strike  Period 
(August  9,  1981,  to 
April  25,  1982) 


3.883 


8.301 


12.184 


This  approach,  however,  does  not  deal  with  the  notion  that  reduced  operational 
levels  normally  lead  to  reduced  rates  of  conflict  (operational  errors  per  mil¬ 
lion  operations).  Therefore,  we  postulated  a  predictable  relationship  between 
operational  errors  and  volume  of  operations  using  statistical  regression  tech¬ 
niques.  We  were  not  able  to  support  the  postulation,  however,  because  the 
available  data  for  errors  and  operations  resulted  in  the  generation  of  inade¬ 
quate  correlation  coefficients.  Nonetheless,  we  were  able  to  determine  that 
the  post-strike  error  rates  were  significantly  lower  than  the  pre-strike  rates. 

In  summary,  our  analyses  showed  the  following: 

•  There  were  fewer  operational  errors  in  the  post-strike  period  than  in 
fh*»  pre-strike  period,  both  for  terminals  and  air  route  traffic  con¬ 
trol  centers. 

•  The  rate  of  operational  errors  (per  million  operations)  was  lower  in 
the  post-strike  period  than  in  the  pre-strike  period,  both  for  termi¬ 
nals  and  air  route  traffic  control  centers. 

•  The  difference  in  the  operational  error  rates  between  the  pre-strike 
and  post-strike  study  periods  could  not  be  attributed  to  change. 

The  findings  support  the  view  that  the  national  airspace  system  is  at 
least  as  safe  in  the  post-strike  period  as  in  the  pre-strike  period. 
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1.0  BACKGROUND 


1.1  History 

On  August  3.  1981 »  a  strike  by  the  air  traffic  controllers  against  the 
FAA  resulted  in  the  loss  of  nearly  11,400  of  the  nation's  air  controller  work 
force,  at  that  time  totalling  about  17,275  persons.  In  the  weeks  and  months 
following  the  strike,  there  was  concern  in  the  aviation  community  as  to  whether 
there  was  safety  degradation  in  the  nation's  air  terminals  and  air  route  traf¬ 
fic  control  centers. 

On  December  8,  1981,  the  National  Transportation  Safety  Board  (NTSB) 
released  a  special  report^-  on  post-strike  air  safety.  The  report,  titled 
"Special  Investigation  Report  -  Air  Traffic  Control  Systems,"  stated  that  the 
ATC  system  "was  operated  safely  in  the  two  months  following  the  strike,"  not¬ 
ing  that  operational  errors  decreased  in  the  post-strike  period.  Moreover,  it 
stated  that  "no  basic  ATC  procedures  were  changed  or  compromised  in  order  to 
keep  the  ATC  system  in  operation,  and  the  high  level  of  ATC  safety  required  is 
possible  within  the  present  system  and  will  be  possible  as  the  system  is 
rebuilt." 

Another  independent  investigation  conducted  by  the  Flight  Safety  Founda¬ 
tion  (FSF)  basically  supports  the  conclusions  of  the  NTSB.  The  FSF  was  com¬ 
missioned  by  the  Office  of  the  Administrator  (FAA)  in  August  1981  to  make  a 

2 

safety  appraisal  of  the  post-strike  ATC  system.  Its  findings,  released  on 
January  29,  1982,  concluded  that  the  ATC  system  as  it  existed  from  September 

^National  Transportation  Safety  Board,  "Special  Investigation  Report  -  Air 
Traffic  Control  System,"  NTSB-SIR-81-7 ,  Washington,  DC,  December  1981. 

2 

Flight  Safety  Foundation,  "A  Safety  Appraisal  of  the  Air  Traffic  Control 
System,"  Report  No.  FSF-ATC-1142-8-82U.  Contract  No.  DTFA01-81-C-10109k 
January  29,  1982. 
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through  November  1981,  maintained  a  level  of  safety  equivalent  to  that  of  the 
pre-strike  system. 

1.2  Purpose  of  this  Study 

The  FAA  Office  of  Aviation  Safety  sponsored  this  study  aimed  at  the  quan 
titative  assessment  of  the  level  of  safety  of  the  air  traffic  control  (ATC) 
system  before  and  after  the  August  3,  1981,  strike.  A  second  objective  was 
to  analytically  compare  those  assessments  in  an  effort  to  determine  whether 
any  degradation  in  the  post-strike  ATC  environment  could  be  observed.  The 
ultimate  goal  of  this  work  is  either  to  confirm  the  earlier  independent 
efforts  or  to  pinpoint  system  weaknesses  (if  they  exist)  and  consider  alter¬ 
native  actions  for  remediation. 

1.3  Alternative  System  Parameters  for  Potential  Use  in  Assessing  the  Safety 
of  the  ATC  System 


A  number  of  alternative  system  parameters  come  to  mind  in  any  discussion 
of  the  safety  of  the  ATC  system.  These  include  operational  errors,  near  mid¬ 
air  collisions,  mid-air  collisions,  and  system  delays. 

"Operational  Errors"  was  chosen  as  the  most  likely  surrogate  to  analyze 
the  ATC  system  since  it  directly  bears  on  safety,  reflects  controller  perfor¬ 
mance,  has  readily  available  pre-strike  and  post-strike  data,  and  is  restricted 
in  where  it  occurs  to  only  "controlled"  airspace.  No  other  alternative  para¬ 
meters  satisfied  all  of  these  requirements  to  the  same  extent  as  did  "opera¬ 
tional  errors." 

1.4  Reporting  Operational  Errors 


The  FAA  defines  an  operational  error  as  an  occurrence  which  results  in 


less  than  the  applicable  separation  minima  between  two  or  more  aircraft,  or 
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between  an  aircraft  and  terrain  or  obstacles  (e.g.,  vehicle  on  runway).  These 
separation  minima  are  defined  in  the  FAA  Handbook  7110.65  or  its  supplemental 
instructions. 

Whenever  an  operational  error  is  thought  to  have  occurred,  authorized 
FAA  personnel  are  required  to  report  the  incident  immediately  to  their  super¬ 
visor.  Once  reported,  the  incident  is  investigated  to  determine  the  cause  or 
reason  for  the  error  and  is  classified  within  one  of  three  categories:  (a) 
human  error,  (b)  procedural  error,  or  (c)  equipment  error. 

The  report  resulting  from  the  investigation  is  reviewed  by  the  Regional 
Air  Traffic  Division  and  is  then  forwarded  to  the  Air  Traffic  Service  Evaluation 
Group,  AAT-20 ,  located  in  Washington,  DC.  This  AAT-20  group  compiles  the 
occurrences  of  operational  errors  on  a  daily  basis  and  maintains  a  file  on  all 
operational  error  reports. 


2.0  ANALYTICAL  METHODS 


2. 1  Operational  Error  Magnitudes  and  Rates 

Data  were  first  collected  on  the  frequency  of  operational  errors,  both 
before  and  after  the  August  3,  1981,  strike.  This  information  was  obtained 
for  terminals  and  centers  separately.  Additionally,  operational  activity 

levels  were  identified  for  the  same  pre-  and  post-strike  periods.  Initially, 
the  data  were  collated  by  month  and  analyzed ,  but  the  analysis  did  not  yield 
conclusive  results  (see  Appendix  B).  The  data  were  collated  by  week,  the 
summary  of  data  consisting  of  54  weeks  of  the  pre-strike  period  and  38  weeks 

of  the  post-strike  period.  Tables  2-1  and  2-2  summarize  these  raw  data. 

From  these  tables  we  find  the  following: 

•  The  magnitude  of  operational  errors  before  the  strike  at  the  terminal 
T 

areas  [ E  E(2-i)^  exceeded  the  magnitude  of  operational  errors  after 
the  strike  at  those  same  locations  for  an  equal  period  of  time 


U 


1* 

E(2-2) 


•  Similarly,  the  magnitude  of  operational  errors  before  the  strike 

Q 

at  the  20  contiguous  centers  [  l  E^  1  exceeded  the  magnitude  of 
operational  errors  after  the  strike  at  those  same  locations  for  an 
equal  period  of  time  [1  E(2-2)^‘* 


Since  the  number  of  operations  was  also  down  at  these  terminals  and  centers, 
such  results  were  not  unexpected  or  conclusive  in  themselves.  The  change  in 
"rate"  of  operational  errors  was  therefore  analyzed  next.  The  following 
results  were  obtained. 

•  The  rate  of  operational  errors/million  operations  was  down  in  the 
terminal  areas  in  the  post-strike  period  compared  to  the  pre-strike 
period: 

*  E  =  Operational  Error,  T  =  Terminal,  C  =  Center 


54 -Week  Pre-Strike  Period 
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Grand  Total  I  E  =  387  Z  (Ops)  =  E  E  -  243  Z  (Ops)  = 

66,378,985  19,250,678 
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139 

35.801 


3.883  versus 


278 

45.342 


•  Similarly,  the  rate  of  operational  errors/million  operations  was  down 
in  the  20  centers  during  the  post-strike  period  compared  to  the  pre¬ 
strike  period: 


Z  E 


(2-2) 


Z  E 


versus - 


(2-1) 


]' 


99 


Z  Ops 


(2-2) 


Z  Ops 


11.926 


8.301  versus 


171 


13.367 


-  12.793] 


(2-1) 


Even  with  such  reductions  in  operational  error  rates,  one  could  still  argue 
that  conclusions  regarding  the  comparative  safety  of  the  ATC  system  could  not 
be  made.  The  argument  goes  something  like  this:  When  a  significant  reduction 
in  operations  takes  place,  then  the  "expected  number"  of  conflicts  (operation¬ 
al  errors)  also  goes  tiown,  but  not  proportionally  —  it  goes  down  geometrically. 
Therefore,  the  next  step  is  to  set  out  to  establish  a  predictable  relationship 
between  errors  and  operations  using  regression  techniques. 

2.2  Regression  Techniques  to  Establish  a  Relationship  Between  Errors  and 

Operations _ _ 

The  approach  adopted  for  the  analysis  of  the  operational  error  data  was 
based  on  the  primary  assumption  that  the  number  of  reported  operational  errors 
will  decrease  with  a  decrease  in  the  number  of  operations  (exposure).  The 
analysis  attempts  to  relate  volume  of  operations  and  operational  errors  for 
both  pre-strike  and  post-strike  periods  by  computing  the  predicted  operational 
errors  for  the  post-strike  period  (based  on  pre-strike  observations)  and 
comparing  the  predictions  with  the  reported  post-strike  errors. 

*  Ops  *  Number  of  Operations  in  Millions 
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If  the  predicted  operational  errors  are  statistically  equal  to  or  above 
the  error  levels  observed  during  the  post-strike  period,  then  one  could  con¬ 
clude  that  safety  was  reasonably  maintained  following  the  controllers'  strike. 
Through  plotting  the  data,  a  preliminary  examination  of  points  revealed  that 
the  relationship  between  operational  errors  and  number  of  operations  appear 
to  be  linear.  This  finding  is  consistent  with  an  earlier  study  by  Lyman^  in 
which  a  simple  linear  regression  using  the  formulation: 

y  =  b  +  b,x 
1  o  1 

showed  a  high  correlation  square  (R2)  between  operational  errors  (y)  and  the 
predictor  variable  x  (number  of  operations)  with  the  value  of  the  F  ratio 
shown  to  be  highly  significant. 

The  analysis  of  the  operational  error  data  as  a  function  of  the  number 
of  operations  handled  by  each  of  the  two  control  areas  was  due  to  the  con¬ 
straints  imposed  by  the  data.  There  were  no  other  data  whose  variability  was 
considered  to  have  a  significant  effect  on  the  number  of  reported  operational 
errors.  In  other  words,  reliable  data  on  other  potential  contributing  factors 
were  not  available.  Thus,  the  initial  analytical  approach  was  to  use  a  simple 
linear  regression  model  of  the  form:  Y  -  a  +  bX.  This  was  done  using  the 
logarithmic  transformed  data. 

However,  the  results  of  these  simple  linear  regressions  gave  reason  to 
attempt  further  analysis  by  regressing  the  data  in  time  lag  periods  (i.e.f 
t,  t-1,  t-2,  and  t-3)  such  that  an  estimate  of  parameters  might  be  obtained 


Lyman,  E.G.,  "ATC  Contingency  Operations  in  the  En  Route  Flight  Regime 
NASA  CR-(166231),  Battelle  Memorial  Institute,  1981. 
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using  an  autoregressive  procedure.  The  model  used  In  this  autoregression 
contains  a  time-series  part  as  well  as  a  systematic  (ordinary  least-squares) 
part  and  approximates  the  following  formulation: 

Et  =  bVt  +  «t 

where 

“t  =  £t  ‘  al  “t-l  ■  a2  “t-2  '  a3  °t-3 


The  error  term  of  this  linear  model  is  assumed  to  follow  a  third  order  auto¬ 
regressive  process.  The  error  factor,  e,  is  assumed  to  be  normally  and  inde¬ 
pendently  distributed  with  mean  zero  and  variance  (o2). 

On  obtaining  the  results  of  the  autocorrelation  using  the  linear  auto¬ 
regression  model,  the  output  is  used  to  estimate  parameters  which  are  linear. 
This  output  was  used  in  the  nonlinear  regression  to  achieve  an  estimate  of 
the  residual  sum  of  squares  between  the  predicted  and  observed  data  points. 
The  details  of  the  analytical  model  are  shown  in  Appendix  A. 

2.3  Test  of  Significance  of  Proportion 

In  comparing  the  data  from  the  pre-strike  and  post-strike  periods,  it 
becomes  important  to  make  statistical  comparisons  between  the  proportions 
associated  with  the  populations  of  the  two  periods.  The  population  propor¬ 
tion  used  for  comparison  of  these  periods  is  the  operational  error  rate, 
expressed  as  the  proportion  of  operational  errors  per  number,  of  operations. 

These  operational  error  rates  are  computable  for  each  observation  and 
for  each  set  of  observations  (i.e.,  towers  and  centers,  pre-strike  and  post¬ 
strike).  The  difference  between  any  observed  proportions  (e.g.,  p^  -  P2)  of 
two  independent  samples  can  then  be  compared  to  determine  whether  or  not  the 
proportions  (i.e.,  error  rates)  for  the  pre-strike  and  post-strike  periods 


came  from  the  same  population  (i.e.,  the  differences  between  rates  were  due  to 
random  chance) . 

For  sufficiently  large  populations  (n^,  n^  >30),  the  sampling  distribu¬ 
tion  of  p^  -  p2  is  approximately  normally  distributed  (and  the  covariance 
between  the  errors  and  the  volume  of  operations  can  be  assumed  to  be  zero) 
with  a  mean  equal  to  p^  -  p^  and  a  standard  error  of: 


+  ?2  11 

tit-)  <q>  ( +  -r  > 

*  1^2 


Therefore,  when  the  null  hypothesis  is  that  of  "no  difference"  between  the 
two  populations  to  be  tested,  the  test  statistic  will  be  Fisher's  "z"  deter¬ 
mined  from  the  formula: 


Thus,  the  computation  of  rates  will  determine  whether  the  operational 
errors  from  the  post-strike  period  normalized  to  the  volume  of  operations  are 
the  "same  as"  or  "lower  than"  the  normalized  operational  errors  reported  from 
the  pre-strike  period,  expressed  with  an  acceptable  level  of  significance. 
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3.0  ANALYSIS  OF  WEEKLY  DATA 

3.1  Data  Reduction 

As  an  attempt  to  improve  the  results  derived  earlier  from  the  monthly 
summary  data,  it  was  decided  to  examine  the  operational  errors  data  using 
expanded  weekly  counts.  Hence,  the  daily  records  of  operational  errors  and 
operations  handled  were  summarized  into  weekly  compilations  from  airport 
towers  (terminals)  and  air  route  traffic  control  centers  (ARTCC)  for  both 
the  pre-strike  and  post-strike  periods.  The  new  summary  (presented  in 
Tables  2-1  and  2-2)  consisted  of  54  weeks  in  the  pre-strike  period  and  38 
weeks  in  the  post-strike  period  (where  additional  data  from  March  and  April 
1982  were  available) . 

The  pre-strike  data  used  in  this  new  summary  consist  of  54  weekly  fre¬ 
quency  counts  of  operational  errors  and  operations  reported  from  July  20, 
1980,  through  August  1,  1981.  These  pre-strike  weeks  were  taken  from  Sunday 
through  the  following  Saturday  (i.e.,  a  normally  defined  week).  However, 
since  the  strike  began  on  Monday,  August  3,  1981,  the  post-strike  weeks  were 
taken  from  Monday  through  the  following  Sunday,  encompassing  a  post-strike 
period  from  August  3,  1982,  through  April  25,  1982.  The  data  collected  on 
Sunday,  August  2,  1981,  were  eliminated  from  the  summary.  The  plots  of 
operational  errors  by  week  are  shown  in  Appendix  E. 

3.2  Linear  Regression  Analysis 
3.2.1  Transformations 

Regression  analysis  was  performed  on  the  weekly  operational  errors  data, 
using  as  many  as  five  different  transformations  of  the  data.  Transformations 
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were  used  to  change  the  original  variable  (particularly  the  independent  variable) 
to  new  variables  for  which  ‘‘he  standard  regression  assumptions  are  more  closely 
satisfied. 

The  three  primary  reasons  for  using  data  transformations  are  (1)  to 
stabilize  the  variance  of  the  dependent  variable  when  the  homogeneity 
assumption  is  violated;  (2)  to  normalize  (i.e.,  to  transform  the  normal  dis¬ 
tribution)  the  dependent  variable  when  the  normality  assumption  is  noticeably 
violated;  and  (3)  to  linearize  the  regression  model  when  the  original  data 
suggests  a  model  that  is  nonlinear  in  either  the  regression  coefficients  and/ 
or  the  original  variables  (dependent  or  independent) .  Very  often  the  same 
transformation  will  simultaneously  help  to  accomplish  the  first  two  foals  and 
sometimes  the  third. 

The  types  of  transformation  used  ares 

a. .  Log  transformation  (y  ■  log  x) :  used  when  the  relationship  of  depen¬ 

dent  variable  (operational  errors)  with  the  independent  variable 
suggests  a  model  with  a  consistently  increasing  slope. 

b .  Reciprocal  transformation  (y  *  i) :  used  to  minimize  the  effect  of 
large  values  of  dependent  variable,  since  the  transformed  values 
will  be  close  to  zero,  and  large  increases  in  the  dependent  variable 

will  only  cause  trivial  decreases  in  transformed  values. 

*1  v 

c.  Arcsin  transformation  (sin  (£)*x):  used  when  a  dependent  vari¬ 
able  is  a  proportion  or  rate,  in  this  case,  the  operational  error 
rate  (i.e.,  errors  divided  by  operations). 
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d.  Square  transformation  (y2  =  x) :  used  when  a  dependent  variable 


increases  with  a  decreasing  slope. 

e.  Constant  multiplicand  transformation  (2y  *  x) :  used  when  the  depen¬ 
dent  variable  increases  at  a  constant  proportion  to  the  independent 
variable  (in  this  case,  the  constant  is  2). 

3.2.2  Results  of  the  Pre-Strike  Data 

A  simple  regression  analysis  was  performed  on  the  various  transformations 
of  the  data,  using  the  general  linear  formulation: 

error  =  g  +  (a)  operations 

where  "6"  is  the  intercept  at  zero  operations  and  "a"  is  the  slope.  (See 
Appendix  D.)  The  results  of  these  regressions  are  presented  in  Table  3-1, 
which  includes  the  correlation  coefficients  (r)  and  other  important  statistics 
for  both  towers  and  centers  for  the  pre-strike  and  post-strike  periods. 

The  pre-strike  untransformed  data  for  towers  yielded  a  correlation  co¬ 
efficient  (r)  of  0.2495  and  for  centers  an  "r"  of  0.1024,  neither  of  which 
were  significant  at  the  0.05  level. 

In  general,  the  various  transformations  did  not  substantially  improve 
these  results.  The  log-transformed  data  for  towers  yielded  an  "r"  of  0.1285 
and  for  centers  an  "r"  of  0.2966.  This  correlation  for  the  centers  data  was 
shown  to  be  significant  at  the  0.05  level,  indicating  that  for  the  center  data 
the  relationship  between  the  operational  errors  and  volume  of  operations  is 
barely  predictable  using  a  log-linear  model.  For  the  centers  data,  the  volume 
of  operations  is  much  less  than  for  the  towers  data  and  the  log  transformation 
would  alter  the  variance  such  that  the  coefficient  of  correlation  would  improve 


Significant  at  0.05  level 
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The  reciprocal  transformation  was  shown  to  improve  the  correlation  for 
both  the  terminal  and  the  center  data,  "r"  equals  0.2737  and  -0.3035,  respec¬ 


tively.  The  reciprocals  tend  to  minimize  the  variance  in  the  independent 
variable.  This  appears  to  increase  the  covariance  to  variance  ratio,  raising 
the  coefficient  of  correlation.  For  the  center  data,  the  negative  correlation 
and  the  negative  slope  show  that  the  errors  decrease  at  an  increasing  rate  of 
operations. 

The  square  transformation  and  the  constant  multiplicand  transformation 
both  raised  the  coefficient  of  correlation  for  the  center  data  to  a  significant 
level.  And  the  3  is  shown  to  improve  from  the  square  transformation  to  the 
constant  multiplicand  transformation. 

In  summary,  the  center  data  appears  to  show  that  the  relationship  between 
operational  errors  and  the  volume  could  be  predictable.  Such  a  relationship 
is  less  apparent  from  the  tower  data. 

3.2.3  Results  of  the  Post-Strike  Data 

None  of  the  "r"  values  are  significant  in  the  post-strike  period.  The 
reciprocal  transformation  does  show  an  improvement  in  the  "r"  values  but  it  is 
not  sufficient  to  be  significant.  The  lack  of  significant  correlation  suggests 
that  the  relationship  between  the  operational  errors  and  the  volume  of  opera¬ 
tions  is  not  well  defined.  However,  the  proportion  of  errors  per  million 
operations  could  be  tested  to  reveal  whether  or  not  the  rate  of  operational 
errors  could  vary  as  the  volume  of  operations.  A  test  of  significance  of 
proportions  was  performed,  and  the  results  are  shown  in  the  following  section. 


3.3  Test  of  Significance  of  Error  Rates 


Regardless  of  the  fact  that  the  results  of  the  linear  regressions  did 
not  yield  significant  relationships  between  the  two  variables  for  pre-strike 
and  post-strike  data  sets,  a  test  of  significance  of  operational  error  rates 
can  be  made  to  determine  whether  or  not  the  drop  in  the  number  of  observed 
errors  during  the  post-strike  period  is  significant.  Since  the  volume  of 
operations  also  dropped  during  that  post-strike  period,  the  question  becomes 
"Has  there  been  a  proportionate  decrease  in  the  errors  corresponding  to  the 
decrease  in  volume  of  operations?" 

The  error  rate  per  million  operations  would  normalize  the  errors  by  divid¬ 
ing  into  the  appropriate  volume  of  operations.  Again,  the  average  error  rate  in 

the  post-strike  period  is  lower  than  in  the  pre-strike  period.  This  in  itself 
does  not  substantiate  evidence  that  it  might  be  expected  based  on  the  propor¬ 
tion  of  errors  in  the  population.  Further,  it  should  be  shown  that  the  dif¬ 
ference  in  rates  is  the  one  that  can  be  obtained  based  on  the  difference 
between  the  pre-strike  and  post-strike  periods,  and  not  due  to  chance. 

The  test  of  proportions  selected  for  this  comparison  between  populations 
was  Fisher's  "z"  distribution,  in  which  the  parameters  to  be  tested  are  the 
proportions  of  errors  per  volume  of  million  operations  (details  of  this  test 
are  presented  in  Appendix  D). 

The  "z"  test  was  performed  for  the  weekly  data.  The  null  hypothesis 
tested  was  H  :p,  *  p  meaning  no  difference  in  error  rates  between  "before" 

O  D  3L 

and  "after"  the  strike.  The  alternative  hypothesis  tested  ^a:P^)<  Pa»  that  is, 
the  error  rate  is  less  in  the  post-strike  period  than  in  the  pre-strike  period. 


When  the  towers  and  centers  data  were  separately  tested  for  the  differ¬ 
ence  between  the  pre-strike  and  post-strike  periods,  the  values  obtained  were 
3.310  and  11.537,  respectively. 

Since  these  "z"  values  were  significant  at  the  0.01  level,  the  conclusion 
was  to  reject  the  null  hypothesis  and  retain  the  alternative  hypothesis.  The 
result  of  this  test  supports  the  conclusion  that  the  difference  in  error  rates 
between  the  two  periods  is  more  than  what  could  be  expected  by  chance  and  that 
the  system,  as  measured  by  operational  error  rates,  was  safer  in  the  post- 
strike  period  than  in  the  pre-strike  period. 

In  addition,  a  test  was  conducted  to  test  the  difference  in  error  rates 
between  the  towers  and  centers  within  the  same  period  —  pre-strike  and  post¬ 
strike  periods.  In  both  the  pre-strike  and  post-strike  periods,  the  center 
error  rates  were  higher  than  the  tower  error  rates.  In  both,  the  "z"  values 
were  higher,  -3.384  in  the  pre-strike  period  and  -5.267  in  the  post-strike 
period,  thus  resulting  in  rejection  of  the  null  hypothesis.  The  conclusion 
is  that  the  center  error  rates  were  larger  than  the  tower  error  rates  and  the 
difference  is  more  pronounced  in  the  post-strike  period  than  in  the  pre-strike 
period. 


20 


4.0  DISCUSSION  OF  RESULTS 


This  study  attempted  to  establish  the  relationship  between  operations 
and  operational  errors.  While  in  general  there  are  statistical  findings  between 
the  two  variables,  the  present  study,  because  of  a  lack  of  sufficent  monthly 
data,  could  not  firmly  establish  a  relationship.  A  further  detailed  study 
was  undertaken  by  expanding  that  data  (i.e. ,  using  weekly  data  instead  of 
monthly  data)  for  both  the  pre-strike  and  post-strike  periods.  The  impor¬ 
tance  of  adding  more  data  points  to  improve  the  relationship  is  evidenced 
by  an  overall  data  analysis  (which  pooled  pre-  and  post-strike  data  points), 
and  is  presented  in  Appendix  C.  The  rationale  for  pooling  the  data  was  an 
attempt  to  investigate  whether  post-strike  data  caused  any  drastic  change  in 
the  relationship  which  existed  between  operational  errors  and  operations  from 
the  post-strike  period. 

When  the  data  were  regressed  using  an  autoregressive  lag  model,  the 
least-squares  estimates  were  nonlinear.  The  estimates  show  that  the  variables 
are  related  and  that  the  estimates  of  coefficients  are  inversely  correlated. 

This  alone  does  not  categorically  establish  that  the  errors  are  smaller  in 
the  post-strike  period  than  in  the  pre-strike  period,  but  it  does  show  that 
the  residual  errors  are  negative  and  the  observed  errors  then  are  smaller  in 
size  and  distance  from  the  estimated  errors. 

However,  this  study  is  a  result  of  analysis  of  data  only,  and  the 
conclusions  should  be  limited  to  that  extent.  A  study  based  on  a  survey  of 
errors  and  the  various  treatment  effects  (such  as  human  errors,  procedural 
errors,  or  equipment  errors)  probably  reveal  further  cause  and  effect  rela¬ 
tionships  of  operational  errors.  The  study  by  Battelle (Lyman)  established  a 


relationship.  This  study  shows  that  If  a  model  based  on  lagged  regressors  Is 
adopted,  the  least-squares  estimates  would  have  to  be  nonlinear,  and  this 
would  render  a  more  generalized  estimate  of  coefficients  than  ordinary  least- 
squares  estimates. 

Further  analysis  of  the  weekly  summary  data  basically  supported  the 
findings  of  the  analysis  of  the  monthly  summary  data.  In  that  the  coeffi¬ 
cients  of  correlations  were  somewhat  high  enough  to  establish  a  statis¬ 
tically  valid  relationship  between  operational  errors  and  volume  of  opera¬ 
tions. 

When  the  weekly  data  were  analyzed  extending  the  rate  concept  further 
using  the  test  of  significance  based  on  the  "z"  test,  the  results  showed  that 
the  rate  of  the  pre-strike  period  was  larger  than  the  rate  in  the  post-strike 
period  and  that  the  difference  was  statistically  noteworthy.  Thus,  this  test 
provides  a  basis  for  believing  that  the  system  has  fewer  errors  in  the  post¬ 
strike  period  and  can  be  rated  safer  relative  to  the  pre-strike  period. 

However,  it  should  be  borne  in  mind  that  the  error  rate  is  only  one  of 
the  measures  that  is  being  used  in  this  analysis.  If  safety  could  be  measured 
as  a  result  of  multiple  factors,  such  as  operational  errors,  near  mid-air 
collisions,  system  difficulty  reports  and  other  parameters,  then  a  more  com- 
prehesive  performance  measurement  of  safety  performance  of  the  system  could 
be  made,  a  comprehensive  performance  measurement  that  would  facilitate  long¬ 
term  predictions  or  forecasting. 

The  conclusions  reached  by  this  study  are  at  best  tentative,  since  there 
was  no  attempt  to  isolate  the  cause  of  errors  and  the  source  of  variance, 
such  as  type  of  operation.  A  study  of  field  operations  would  be  useful. 
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Moreover,  it  Is  feasible  to  suggest  that  the  working  conditions  surrounding 
the  strike  itself  were  responsible  for  an  apparent  drop  in  operational  error 
reports  for  a  number  of  reasons ,  not  the  least  of  which  could  be  the  existence 
of  unreported  errors  during  the  hectic  period  following  the  strike.  Further, 
it  is  the  opinion  of  some  observers  that  tension  among  ATC  personnel  during 
the  period  preceding  the  strike  could  have  caused  a  higher  reporting  rate  of 
operational  errors  during  that  time. 

In  addition,  the  study  took  into  account  only  one  exposure  variable  — 
namely,  volume  of  operations.  There  are  many  factors  besides  traffic  volume 
(e.g.,  number  of  hours  flown  by  pilot,  aircraft  miles  flown,  number  of  air¬ 
craft  hours  flown)  that  would  have  certainly  influenced  the  number  of  reported 
operational  errors  during  the  period  under  analysis,  and  which  may,  when  used 
singularly  or  in  combination,  offer  an  improved  correlation  with  operational 


errors. 


5.0  CONCLUSIONS  AND  RECOMMENDATIONS 


5.1  Cone''  is  Ions 

In  summary,  the  analysis  o£  the  operational  errors  reported  during  the 
specified  periods  before  and  after  the  August  3,  1981,  air  traffic  control¬ 
lers'  strike  provides  statistical  confirmation  that  national  airway  safety 
was  not  adversely  affected  after  the  strike.  Conclusions  of  the  analysis 
are  as  follows: 

a.  Post-strike  operational  errors  normalized  to  the  volume  of  opera¬ 
tions  (error  rate)  are  shown  to  be  statistically  lower  than  nor¬ 
malized  operational  errors  reported  In  the  period  prior  to  the 
strike,  using  a  test  of  significance  of  proportions. 

b.  Pre-strike  operational  error  data  from  centers  and  towers  appeared 
to  show  some  correlation  (though  the  correlation  is  inadequate 
for  prediction  purposes)  between  operational  errors  and  number  of 
operations  using  a  simple  linear  regression  model  with  specific 
transformations . 

c.  Neither  the  post-strike  operational  error  data  from  the  terminals 
or  those  from  the  centers  appeared  to  show  a  significant  relation¬ 
ship  between  errors  and  operations  using  a  simple  linear  regression 
model. 

5.2  Recommenda  t  ions 

Observations  made  during  the  course  of  this  analysis  have  contributed  to 
the  following  recommendations: 

a.  Useful  by-products  generated  by  this  analysis  are  the  readily  avail¬ 
able  daily  operational  errors  and  corresponding  operation  volume 


data,  which  can  be  used  to  identify  peak  and  nonpeak  levels  of 
operational  error  occurrences  by  such  categories  as  weekdays, 
weekends,  and  holidays,  along  with  monthly,  seasonal,  and  annual 
fluctuations.  In  view  of  the  very  limited  research  done  on  the 
operational  errors  and  operation  activity,  it  would  be  useful  to 
study  and  examine  further  what  hazard  exposure  parameters  affect 
the  occurrence  of  operational  errors.  Since  airway  incident 
reports  (such  as  operational  errors,  aircraft  accidents,  and 
near-miss  collisions)  are  functions  of  various  risk  exposure 
(such  as  aircraft  delay,  number  of  operations,  pilot  hours,  air¬ 
craft  miles,  and  aircraft  hours),  further  analysis  would  serve  to 
identify  and  clarify  exposure/incident  relationships. 

b.  As  independent  test  cases,  certain  towers  and  centers  should  be 
selected  where  the  system  was  stable  (known  technology  coefficient 
or  factors)  and  these  could  be  used  as  a  representative  sample  by 
geographical  location  or  type  of  activity.  These  individual  case 
studies  would  provide  the  insight  and  necessary  steps  to  enlist 
more  representative  samples  for  extrapolation  to  a  national  level  — 
bottom-up.  The  results  of  this  approach  could  be  checked  against 
the  top-down  method  using  predicted  operational  errors  only  based 
on  nationwide  total  statistics. 

c.  The  exponential  relationship  between  operational  errors  and  traffic 
volume  could  be  further  investigated  and  validated.  For  example, 
the  decrease  in  the  number  of  operational  errors  has  been  shown  to 
be  greater  (exponentially  related)  than  the  decrease  in  the  number 
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of  operations  —  meaning  that  if  the  number  of  operations  is  reduced 
by  half,  the  errors  could  be  reduced  by  more  than  half. 

d.  Continue  work  on  operational  error  categorization  which  deals  with 
additional  raw  data.  This  involvement  will  provide  more  expanded 
opportunity  for  safety  analysis. 

e.  Perform  additional  statistical  analysis  and  prepare  associated 
graphical  representations  separately  for  centers  and  towers  as  well 
as  combined  data  with  descriptive  statistics,  such  as  mean,  standard 
deviation,  etc.  As  a  result  of  such  analysis,  it  will  be  possible 
to  make  a  visual  comparison  of  pre-strike  and  post-strike  periods 
using  the  absolute  number  of  operational  errors. 

f.  Enhance  familiarity  with  available  raw  data  and  data  base  functions, 
and  establish  a  monitoring  program  relating  to  any  variation  in  data 
by  group, of  hours,  areas,  mode  of  operation,  etc.  This  effort  must 
also  include  a  readiness  to  undertake  a  limited  full  scale  analysis 
in  the  areas  of  human  errors,  hardware  errors,  and  procedural  errors. 
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APPENDIX  A.  ANALYTICAL  MODELS 


The  model  for  the  simple  linear  regression,  the  autoregressive  model, 
and  the  methods  relevant  to  non-linear  least  squares  estimates  of  the  parameters 
used  are  presented  below: 

A.  1  Linear  Model 

Let  E  denote  the  operational  errors  and  V  the  volume  of  operations,  then 

E  =  0  log  V  +  «.  ^  is  the  model  which  would  describe  the  relationship 

betweeen  the  two  variables,  a,  g  being  coefficients  of  the  model  commonly 

referred  to  as  intercept  and  slope,  respectively. 

In  the  case  of  regression, 

_  _ 

(E  -  E)  =  r  —  (V  -  V) 

av 

_ 

or  E  *  f  —  (V  -  V)  +  E  (2) 

°V 

This  can  be  readily  seen  as  (1)  (the  equation  (2)  reduces  to  (1) ) 

°E  - 

if  r  —  =  6,  (V  -  V)  =  log  V  and  E  “  a 

°V 

which  is  a  linear  function. 

In  the  ordinary  least  squares  (OLS)  estimation  of  parameters,  the  partial 

derivatives  are  minimized.  If  log  V  ■  V*,  then  partial  derivatives  are 

minimized. 

(E  -  8V*  -  a) 2  =  0  (3) 

v 

However,  if  both  the  pre-strike  data  and  post-strike  data  were  put  into  one  data 
base,  the  autocorrelations  have  to  be  verified. 

A-l 


A. 2  Autoregressive  Model 
E  -  Vg 

is  the  basic  model.  When  the  time  series  regression  model  is  assumed,  it  becomes 


Et  =  V  +  at 

where 

a  =  e  -  a. a  . 
t  t  lt-1 


(4) 


a2at-2 


akat-k 


and  et  are  i_  i.  d.  N(0,  o2). 

a^....ak  are  autoregressive  parameters. 

To  obtain  estimates  of  the  parameters  a^....a^  the  following  equations  of 
Yule-Walker  form  are  solved. 


Toep  [rQ . rk-1]  a  =  [r^^. . . . rfc] 


[r^....rk]  is  a  k  x  1  vector  of  autocovariances  and  Toep  is  Toeplitz 
operator. 

All  the  variables  are  transformed  in  the  following  way.  The  first  k 
observations  are  transformed  using  the  form: 


A  X 

op  [V1....Vk] 


The  remaining  observations  are  transformed  using  the  autoregressive  model 


k 

+  Y 

i-i 


ai  Vi 


(5) 


6  the  regression  parameter  that  is  estimated  in  the  OLS  form. 


The  model  used  to  obtain  least-squares  estimates  of  regression  coefficients  is 
in  the  following  form: 

E  =  Bq  +  Bj  (V)  +  Ra  (Et-1  -  Bq  -  Bx  (V^))  (6) 

This  is  nonlinear  in  form.  Therefore .  nonlinear  linear  squares  estimation  is 
needed . 

A. 3  Nonlinear  Estimation  of  Parameters 

As  in  the  linear  model,  the  nonlinear  linear  squares  estimation  proceeds 

with  E  =  F  (B  ,  B,  .  .  .  V, ,  V0  )  +  e  »  F(B)  +  e 
o  I  i  i 

The  normal  equations  are 

V'F  (B)  =  V'E  where  V  -  3F/9B 

The  solution  is  necessarily  iterative  and  after  an  initial  value  is  chosen 
for  B,  and  its  estimated  value  is  improved  until  £'£  (error  sum  of  squares)  is 
minimized. 

The  Gauss-Newton  method  uses  expansion  of  Taylor  series 

F(B)  -  F  (B  +  V  (B  -  B  )  +  V'  (V  -  B.)  +  ...) 
o  o  1 


where 


This  series  substituted  In  the  normal  equations  will  yield 

V'  F  (B)  -  VfE 

V’  [F  (B  )  +  V  (B-B  )]  *  VfE 
o  o 

V'  F  (B  )  +  V'V  (B-B  )  *  VrE 
o  o 

VfV  (B-Bq)  *  VrE  -  VrF(B0) 

-  V'  (E-F  (B  ) 
o 

«  v'e 

(B-B  )  -  (V'7)'Ve. 

o 

A  *  (V'V)-1V'e 

A  .A  A  A 

This  proceeds  until  e'e  (Bq  +  kA)  <  e'e  (Bq)  . 

However,  in  estimating  Bq,B^  in  Equation  (6)*  we  need  a  derivative-free  method, 
because  the  variables  in  the  function  do  not  satisfy  the  properties  of 
continuous  function.  The  algorithm  called  DUD  *  (Doesn't  Use  Derivatives)  was 
used  to  estimate  using  least-squares  procedure.  It  is  essentially  a  modified 
Gauss-Newton  algorithm.  The  procedure  is  as  follows: 

1Ralston,  M  and  Jennrich,  R.  "DUD,  Derivate-Free  Algorithm  for  Non-Linear 
Least  Squares,"  Technometrics ,  February  1978,  p7-14. 
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The  least  squares  begins  with  minimizing  the  sum  of  the  squares  of  a  parameter  B 


F(B) 


n 


-  2  (V.  -  f .  (B))2 
i-1  1  1 


-  tl  v±  -  Fi 


for  n  observations 


(7) 


Step  1.  Let  B™....Bm  n 
1  P  +  1 


p  *  no.  of  parameters  to  be  estimated 
m  ■  no.  of  iterations 


2.  F  (B)  -v  im(B) 

3.  Find  Bm  3  min  D  UB)-E 

4.  Bm'»—  B™  :  i  ■  l....p+l  in  m+1  iterations 

B  =  Bp+1  +  A  B  a,  (8) 


B^,  B^ • .  •  .B  ^  are  estimates  from  prior  iterations. 


The  linear  approximation  is  given  by 


i(a)  *  f  (Bp+^)  +  AFa,  where 


i  th  column,  AFV  -  f(B1>  - 


One  Iteration  consists  of  minimizing  function 


$(<*)  -  (e-i  (a))'  (e-£(a)). 

The  solution  is 

a  -  (Af'aF)"1  AF (e-f  (Bp+1)) 

Then  this  value  is  entered  into  Equation  (8)  to  obtain  B.  This  iteration  stops 
when  ESS  is  nHiHmnm  or  no  reduction  in  ESS  is  found  when  convergence  is  assumed. 
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APPENDIX  B.  ANALYSIS  OF  MONTHLY  DATA 
B.l  Data  Reduction 

The  daily  records  of  operational  errors  reported  separately  from  airport 
towers  (terminals)  and  air  route  traffic  control  centers  (ARTCC)  were  totalled 
for  all  user  categories  (i.e.,  Air  Carrier,  General  Aviation,  Air  Taxi/Commu¬ 
ter,  and  Military).  These  daily  error  counts  were  summed  to  get  monthly  error 
compilations  for  the  19-month  pre-strike  (January  1980  through  July  1981)  and 
the  seven-month  post-strike  (August  1981  through  February  1982)  periods. 

The  number  of  operations  handled  per  month  was  required  for  use  in  the 
analysis.  The  operations  data  were  obtained  from  the  Office  of  Aviation 
Policy  and  Plans  (APO) ,  received  as  daily  frequency  counts  for  combined  user 
types,  from  each  controlled  airspace  (i.e.,  terminals  and  ARTCCs).  The  monthly 
operation  totals  were  compiled  separately  for  the  19  pre-strike  months  and 
the  seven  post-strike  months. 

The  monthly  operation  totals  were  recorded  in  millions  of  operations,  a 
form  that  does  not  offer  much  suitability  for  analysis.  Therefore,  the  totals 
were  re-expressed  before  analysis,  using  the  transformation  function  of  loga¬ 
rithm  to  the  base  of  10  (i.e.,  common  logarithm)  since  it  is  divisible  into 
a  million.  By  transforming  the  monthly  operation  totals  into  this  form,  the 
operational  error  data  and  the  operations  data  take  on  a  more  symnetric  form. 

Tables  B-l  and  B-2  present  the  operational  errors  data  for  the  pre-strike 
and  post-strike  periods,  respectively.  The  pre-strike  data  consist  of  19 
monthly  frequency  counts  of  operational  errors  reported  throughout  the  year 
1980  and  on  up  to  July  1981.  The  post-strike  data  consist  of  seven  monthly 
frequency  counts  of  operational  errors  that  were  reported  from  the  post-strike 


Summary  of  Monthly  Operational  Errors  and  Totals  of  Operations 
for  the  Seven-Month  Post-Strike  Period 


<0 

C 

o  o  <2 

ft  00 
M  U  00 

<u  a  o 

,0  H  <"H 
6  4)^ 
3  0* 

Z  O 


4) 

>u  e  <o 
o  o  u 

•H  O 
huh 
tl  it  h 
ii  h  u 
B  «J 
3  O. 

Z  O 


0) 

h-t  C  ® 
O  O  U 
•H  O 
huh 
<u  «  H 
.O  M  Ed 

i  g. 

z  o 


ON 

ON 

ON 

0- 

CO 

ON 

00 

m 

00 

NO 

o 

NO 

NO 

o 

o 

rH 

ON 

p>. 

o 

ro 

NO 

co 

CO 

co 

CO 

CO 

CO 

CO 

CO 

CO 

CO 

vO 

NO 

NO 

vO 

NO 

NO 

so 

00 

NO 

rH 

ON 

HT 

CO 

m 

H 

rH 

rH 

rH 

m 

» 

rH 

'S' 

'O' 

in 

CO 

00 

CM 

vO 

O' 

NO 

oo 

o 

uo 

o 

CM 

r* 

ns 

-a- 

-<r 

rH 

CO 

VO 

vO 

vO 

vO 

VO 

in 

in 

m 

• 

• 

• 

• 

• 

• 

• 

NO 

vO 

NO 

NO 

vO 

NO 

vO 

1  ^ 

m 

rH 

ON 

o 

CO 

co 

rH 

rH 

04 

rH 

rH 

rH 

rH 

rH 

rH 

rH 

rH 

rH 

CM 

CM 

00 

00 

00 

00 

00 

00 

00 

B-3 


data  of  August  1981  through  February  1982.  The  monthly  counts  are  separated 
for  terminals  (airport  towers)  and  ARTCCs  (cer'ers). 

In  addition  to  the  monthly  operational  errors,  the  tables  provide  the 
monthly  totals  of  terminal  operations  and  ARTCC  operations  expressed  in  their 
common  logarithms  [i.e.,  log  (V),  where  V  is  the  monthly  total  of  operations] . 

On  examining  the  data  provided  in  these  tables,  it  is  evident  that  the 
operational  errors  generally  appear  to  decrease  as  the  totals  of  operations 
decrease,  in  a  rather  monotonic  fashion  for  both  terminals  and  centers.  And 
even  while  the  number  of  operations  per  month  shows  a  decline  after  the  August 
1981  date,  the  relationship  between  the  monthly  operational  errors  and  the 
monthly  operations,  both  for  the  terminals  and  the  centers,  appears  to  have  a 
similar  pattern  to  the  relationship  shown  in  the  pre-strike  period. 

B .2  Linear  Regression  Analysis 
B.2.1  Methodology 

The  initial  analysis  of  the  data  was  made  using  a  simple  linear  regres¬ 
sion  model  of  the  form: 

E±  -  bx  log  (V±)  +  bQ 

where 

is  monthly  operational  errors,  and 
is  monthly  operations  handled. 

In  this  simple  data  fitting  procedure,  using  the  method  of  least-squares  to 
estimate  the  parameter,  the  objective  is  to  find  the  values  of  the  constants 
in  the  above  equation  that  minimize  the  sum  of  the  squared  deviations  of  the 
observed  values  from  those  predicted  by  the  equation. 


The  least-squares  estimates  of  b  and  b,  are  those  values  of  b  and  b.  which 

o  1  o  1 


minimize  the  function: 

N 

Q  -  Z  (E±  -  bx  log  (V±)  -  bo)2 

The  estimates  of  b  and  b.  and  the  estimate  of  the  variance,  a2,  are 
o  l 

obtainable  through  the  use  of  regression  procedures,  using  the  method  of  ordi¬ 
nary  least-squares.  And  later,  using  the  autoregressive  process,  the  auto¬ 
correlation  of  lag  coefficients  and  their  covariance  are  estimable  by  means  of 
an  autoregression  procedure,  which  first  estimates  the  model  E  ■  bV  using  the 
method  of  ordinary  least-squares  and  then  computes  the  autocorrelation  up  to 
lag  «  3  of  the  residuals  from  the  ordinary  least-squares  regression. 

B.2.2  Results  of  Pre-Strike  Period 

The  results  of  the  simple  regression  on  the  19  pre-strike  tower  observa¬ 
tions  using  the  linear  model: 

Errors  *  K  log  (Operations)  +  C 

show  that  only  12  percent  (R2  ■  0.1191)  of  the  variance  in  operational  errors 
is  explained  by  the  linear  model  using  volume  of  operations  as  the  only  pre¬ 
dictor  variable.  This  weak  linear  relationship  is  reflected  by  the  low  value 
(2.30)  of  the  F  ratio,  shown  only  to  be  significant  at  the  0.15  level. 

The  corresponding  results  on  the  19  pre-strike  center  observations,  while 
shown  to  be  slightly  better  than  those  reported  for  the  tower  data,  still  did 
not  evidence  a  strong  linear  relationship  between  operational  errors  and  num¬ 
ber  of  operations.  In  this  case,  20  percent  of  the  variance  in  operational 
errors  could  be  explained  by  the  linear  model  with  the  computed  F  value  of 
4.18  shown  only  to  be  significant  at  the  0.10  level. 


The  best  linear  models  for  each  data  group,  based  on  the  estimates  using 
the  simple  regression  procedure,  were  as  follows: 

For  Tower  Data:  E  =  54.27  log  (Ops)  -  334.47 
For  Center  Data:  E  *  126.45  log  (Ops)  -  790.41 

The  standard  deviation  for  each  group  of  data  (s  =  6.27  for  the  tower 
data  and  5.24  for  the  center  data)  was  found  to  be  quite  large  for  an  n  =  19. 

Based  on  these  results,  the  linear  model  used  for  these  regressions 
appears  to  have  been  insufficient  to  describe  the  data,  so  an  autoregression 
model  using  time  lag  coefficients  was  tried  in  order  to  achieve  a  better  data 
fit. 

Using  a  specified  lag  order  of  t  ■  3,  the  resulting  model  took  the  form: 


Where 

Et  is  the  number  of  operational  errors  at  a  given  time 
is  the  (log)  volume  of  operations  at  a  given  time 

b  is  the  regression  coefficient,  and 

<*t  is  the  autoregression  coefficient. 

The  results  of  the  autogression  on  the  tower  data  using  this  modified 
linear  model  are  presented  in  Table  B-3  and  show  that  the  autocorrelations 
of  the  residuals  from  the  ordinary  least-squares  regression  appear  to  change 
sign  after  the  second  lag  (t-2).  Note  that  in  the  autocorrelation  procedure 
used,  the  second  order  lag  coefficient  was  set  equal  to  zero  (i.e. ,  &2  -  0). 
And,  the  covariance  of  the  estimated  b  values  with  Et  is  shown  to  be  0.05. 


Table  B-3.  Results  of  Autoregression  on  the  Pre-Strike  Tower  Data 


Model 


:  Et  “  bVt  +  et  ”  alat-l  ”  a2at-2  “  a3at-3 


Or d inary  Least  Squares  Estimates 


Variable 

(Intercept) 


Degree  of  Freedom 


b  Value 


4.5670 


2.  Estimates  of  Autocorrelations 

Lag  Covariance  Correlation 


39.1352 

-7.0614 


-7.3376 


7.7618 


1.0000 


-0.1804 

-0.1875 


0.1983 


Bar  Chart  of  Autocorrelations 
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Table  B-3  (Continued) 
Estimates  of  the  Lag  Coefficients 


Coefficient 


Lag 

1  0.1484 

3  -0.1704 


Std.  Deviation 

0.2263 

0.2263 


Error  Terms 


Sum  of  Squares  (SSE)  670.5215 

Deg.  of  Freedom  (DFE)  17 

Mean  Square  (MSE)  39.4424 

Root  Mean  Square  (Root  MSE)  6.2803 


Estimate  of  b-Value 

Variable  DF  b-Value 

Intercept  0  0 

V  1  4.5830 


Std.  Deviation  t  Ratio 

0 

Q. 2209  20.743 


Covariance  of  b-Value 


Intercept 


Intercept  V 

0 


0.0001 


0 


V 


0 


0.0488 


The  estimates  for  the  time-lag  coefficients  produced  the  model: 

'■  *  e  -  0.0474  at_-^  +  0.0458 

Terforming  similar  procedures  on  the  pre-strike  center  data  (see  Table 
B-4) ,  the  results  of  the  systematic  part  of  the  autoregressive  process  showed 
the  covariance  of  the  b  values  with  Et  to  be  0.04. 

And  the  results  of  the  time-series  part  yielded  an  autogression  coeffi¬ 
cient  of: 

a  ■  e  -  0.0474  at_^  +  0.0458  at_^ 

B.2.3  Results  for  Post-Strike  Period 

When  the  post-strike  data  were  analyzed  using  the  simple  linear  regres¬ 
sion,  neither  the  tower  data  nor  the  center  data  showed  a  strong  linear  rela¬ 
tionship  between  the  two  parameters.  For  the  tower  data,  about  45  percent 
(Ra  ■  0.4539)  of  the  variance  in  operational  errors  could  be  explained  by 
the  model  using  volume  of  operations,  while  only  0.2  percent  (R2  =  0.0022) 
of  the  variance  in  center  operational  errors  were  explainable  using  the  volume 
of  center  operations. 

These  findings  were  reflected  by  their  corresponding  low  F  values,  which 
were  significant  only  at  the  90  percent  and  the  10  percent  confidence  levels, 
respectively. 

This  weak  linearity  could  have  been  expected  since  the  pre-strike  data 
did  not  fit  a  similar  linear  model.  However,  because  the  post-strike  data 
only  consists  of  five  observations  (after  deducting  the  two  degrees  of  free¬ 
dom)  ,  it  could  not  be  examined  as  a  separate  universe  by  applying  autocorre¬ 


lation  procedure. 


Table  B-4.  Results  at  Autoregression  on  the  Pre-Strike  Center  Data 


Model:  Efc  -  bVfc  +  ^  -  a^  -  a^  -  a^ 


Ordinary  Least-Squares  Estimates 


Variable 


DF 


b-Value 


(Intercept) 

V. 


0 

1 


0 

2.9613 


Estimates  of  Autocorrelations 


Lag 

Covariance 

Correlation 

0 

30.2969 

1.0000 

1 

-1.4066 

-0.0464 

2 

0.6644 

0.0219 

3 

1.3574 

0.0448 

Bar  Chart  of  Autocorrelations 
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Table  B-4  (Continued) 


3.  Estimates  of  the  Lag  Coefficients 


Coefficient 


Std.  Deviation 


t  Ratio 


0.0474 

-0.0458 


0.2289 

0.2289 


0.2071 

-0.2002 


Error  Terms 

Sum  of  Squares  (SSE)  571.1357 

Deg.  of  Freedom  (DFE)  17 

Mean  Square  (MSE)  33.5962 

Root  Mean  Square  (Root  MSE)  5.7962 

4.  Estimate  of  b-Value 


Variable 


b-Value 


STD  Deviation  t  Ratio 


Approx.  Prob. 


(Intercept)  0 


2.9649  0.2072 


14.304 


0.0001 


5.  Covariance  of  b-Value 


Intercept 


Intercept 


0.0429 


w 


B.3  Non-Linear  Regression  Analysis 


B.3.1  Methodology 

After  estimating  lagged  independent  regressions,  the  least-squares  esti¬ 


mators  were  used  to  estimate  the  residuals  between  the  predicted  and  observed 


data  points.  This  comparison  of  residuals  will  serve  to  illustrate  how  well 
the  residuals  converge  to  give  good  estimates  of  the  parameters. 


While  there  are  many  iterative  methods  that  could  be  used  to  obtain  non¬ 
linear  least-squares  estimators,  the  method  selected  for  this  data  analysis 
was  a  derivative-free  method.  This  method,  described  in  detail  in  Appendix  A, 
uses  a  modified  "Gauss-Newton"  method  (to  converge  on  a  solution)  so  that  both 
the  size  and  the  direction  of  steps  can  be  determined  simultaneously. 

The  model  used  in  the  iterative  method  is  tthe  following: 

E  -  b0  +  b,  (V)  +  r  [Vt_,  -  b„  -  b,  ] 

where 

bQ  and  b^  are  regression  coefficients 
V  2  is  operations  lagged  by  t-1 

E  j  is  operational  errors  lagged  by  t-1,  and 

r  is  correlation  coefficient. 


The  output  of  the  nonlinear  regression  analysis  will  produce  a  set  of 
residual  sum  of  squares  which  will  give  some  idea  of  how  well  the  data  fits 
the  predicted  model  values.  These  results  would  then  give  an  indication  of  the 
size  of  the  residual  errors  as  a  test  of  the  estimated  parameters. 


To  obtain  these  results,  the  nonlinear  regression  procedures  were  used 
to  produce  least-squares  estimates  of  the  parameters  of  the  nonlinear  model 
presented  earlier.  The  procedure  first  performs  a  grid  search  to  determine 
starting  values  for  the  parameters  to  be  estimated,  and  then  uses  the  pre¬ 
scribed  iterative  method  to  obtain  its  nonlinear  least-squares  estimations. 

The  method  regresses  the  residuals  on  the  partial  derivatives  of  the  model 
with  respect  to  the  parameters,  until  the  iterations  converge. 

B .  3 . 2  Results  for  Pre-Strike  Period 

Using  the  first  order  lag  coefficients  for  both  variables,  a  nonlinear 
least-squares  estimation  was  attempted  using  the  modified  Gauss-Newton  itera¬ 
tion  described  above. 

The  results  of  this  nonlinear  regression  procedure  on  the  pre-strike 
tower  data  yielded  the  following  model: 

errors  =  -27.6738  +  8.8218  (tops*)  +  r[(tops*)  ,  +  27.6738  -  8.8218  (errors)  ] 

t-1 

with  the  asymptotic  correlation  (r)  between  the  regression  coefficients,  bQ, 
and  b^  (refer  to  model),  shown  to  be  -0.9988,  a  strong  negative  correlation. 

The  residual  sum  of  the  squares  produced  by  the  derivative-free  model  showed 
that  the  coefficients  converge  at  the  end  of  the  third  iteration.  And,  the 
residual  errors  which  are  produced  by  taking  the  difference  between  the 
observed  and  the  predicted  values  of  dependent  variable  (operational  errors) 
fall  close  within  a  range  of  ±12. 

For  the  pre-strike  center  data,  the  nonlinear  least-squares  estimations 
using  the  first  order  lag  coefficients  for  each  variable  produced  the  model: 

*  Note:  The  notation  "tops"  in  the  above  equation  is  the  log  of  the  tower 
operation. 


errors  -  222.7691  +  37.6894  (cops*)  +  r[(cops*)  +  222.7691-  37.6894  (errors) 

C~1  t 

with  the  asymptotic  correlation  (r)  between  the  regression  coefficients,  bQ, 
and  b^,  shown  to  be  -0.9999,  a  very  strong  negative  correlation.  The  residual 
sum  of  the  squares  shows  a  convergence  of  coefficients  after  the  third  itera¬ 
tion.  And,  the  residual  errors  fall  close  within  a  range  of  ±8. 

The  successful  application  of  this  nonlinear  least- squares  estimation 
procedure  is  not  possible  with  the  seven-point  post-strike  data  as  stated 
in  the  previous  Section  B.2.3. 

B.4  Comparison  of  Pre-Strike  and  Post-Strike  Results 

In  attempting  to  compare  the  results  between  the  pre-strike  and  post¬ 
strike  data  analysis,  it  is  important  to  be  aware  of  limitations  associated 
with  such  a  comparison:  the  seven  post-strike  observations  are  considered 
only  a  small  sample.  With  this  condition  stated,  it  was  found  that  when 
comparing  the  sum  of  the  squares  across  the  products  matrix,  the  cross- 
products  between  operational  errors  and  operations  were  higher  for  the  post¬ 
strike  data  than  the  pre-strike  data.  The  sum  of  the  squares  of  operations 
is  almost  50  percent  of  the  cross-products.  The  variability  is  somewhat 
higher  in  the  post-strike  data  in  the  independent  variable. 

In  the  centers'  data,  the  R2  in  the  pre-strike  data  was  0.1973,  barely 
significant  at  0.05  level.  The  "t"  ratio  of  the  parameter  estimate  was  2.044, 
while  in  the  post-strike  data,  the  R2  was  only  0.0022.  The  cross-product  is 
higher  proportionately  in  the  post-strike  data  than  in  the  pre-strike  data. 


*  Note:  The  notation  "cops"  in  the  above  equation  is  the  log  of  center 
operations. 


PLOT  OF  ERRORS«QPERNS  SYMBOL  IS  VALUE  OF  MONTH 


Figure  B~l.  Relationship  Between  Tower  Operational  Errors  and  Number  of 
Operations  Handled  (Pre-Strike  Period) 


PLOT  OF  ERRORS«COPERNS  SYMBOL  IS  VALUE  OF  MONTH 


Figure  B-2 .  Relationship  Between  Center  Operational  Errors  and  Number 
of  Operations  Handled  (Pre-Strike  Period) 


APPENDIX  C.  OVERALL  DATA  AND  TIME  SERIES  ANALYSIS 


The  objective  of  pooling  pre-  and  post-strike  data  and  conducting  overall 
time  series  analysis  is  to  examine  whether  there  is  any  difference  of  relation¬ 
ship  between  operational  errors  and  operations  in  the  two  periods.  Note  that 
this  is  only  an  indirect  method  of  testing  the  difference  existing  in  two  per¬ 
iods,  and  is  undertaken  only  because  there  was  a  lack  of  sufficient  data  in  the 
post-strike  period.  If  the  difference  in  results  obtained  form  overall  data  and 
pre-strike  data  analysis  is  negligible,  then  it  suggests  that  the  pre-strike 
and  post-strike  systems  are  alike  and  similar  in  safety.  Both  simple  regres¬ 
sion  and  autoregression  models  were  fitted  and  analyzed  with  overall  data,  the 
results  of  which  are  presented  below: 

C. 1  Overall  Data  Investigation 

The  results  of  the  simple  regression  on  the  overall  data  group  showed 
improvement  in  R2  when  compared  to  the  regression  results  of  the  individual 
data  groups.  For  the  tower  data,  58  percent  (R2  ■  0.5779)  of  the  variance  in 
operational  errors  can  be  explained  using  the  linear  regression  model,  with  a 
high  value  of  32.86,  shown  to  be  significant  at  the  0.0001  level. 

Correspondingly,  nearly  45  percent  (R2  ■  0.4474)  of  the  variance  in  center 
operational  errors  are  explainable  using  the  linear  model,  with  the  associated 
F  ratio  of  19.43  having  a  0.0002  level  of  significance. 

The  linear  models  for  each  data  group  corresponding  to  their  estimates 
of  parameters  determined  from  the  single  regressions,  are  as  follows: 

For  Tower  Data:  E  *  98.01  (Ops)  -  629.65 

For  Center  Data:  E  *  130.01  (Ops)  -  813.31 
Note  that  the  model  particular  to  the  center  data  appears  to  be  comparable 

to  that  model,  shown  earlier,  from  the  pre-strike  data  regression. 


C-l 


Upon  using  the  autoregression  procedure  on  the  overall  tower  data,  the 
resulting  autocorrelations  (Table  C-l)  all  were  found  to  be  positive  (±) ,  and 
to  improve  after  the  third  order  lag  (t*3).  The  estimate  of  the  b  coefficient 
in  the  autoregression  model,  found  to  be  3.7834,  is  shown  to  be  significant  at 
the  0.0001  level  using  the  t  ratio.  And,  the  covariance  of  the  estimated  b 
value  with  the  dependent  variable  (Et)  is  shown  to  be  0.21. 

Applying  the  nonlinear  derivative-free  procedure,  using  the  first  order 
lag  coefficients  determined  in  the  preceding  autoregression,  the  estimates 
of  the  parameters  appear  to  converge  after  the  third  iteration.  The  result¬ 
ing  model  determined  by  the  nonlinear  regression  was: 

errors  -  0.8239  +  4.0188  (tops)  +  r  I(tops)t_^  +  0.8239  -  4.0188  (errors)t_1l 
with  the  asymptotic  correlation  (r)  between  the  regressive  coefficients,  bQ 
and  b^,  shown  to  be  -  0.9067. 

The  residual  errors  were  within  a  range  of  ±10,  indicating  that  the 
predicted  model  data  was  close  to  the  actual  observations.  And  further, 
on  examining  the  residual  errors  in  the  post-strike  portion  of  the  combined 

data,  the  values  of  the  observed  operational  errors  were  consistently  lower 
than  the  predicted  operational  errors  as  evidenced  by  the  residuals. 

Less  dramatic  results  were  shown  with  the  center  group  of  overall  data 


(see  Table  C-2). 


Table  C-l  (Continued) 


3.  Estimates  of  Lag  Coefficients 

Lag  Coefficients 

1  -0.3168 

3  -0.3234 


Std.  Deviation  t  ratio 

0.1758  -1.8017 

0.1758  -1.8394 


Error  Terms 


Sum  of  Squares  (SSE) 

Deg.  of  Freedom  (DFE) 

Mean  Square  (MSE) 

Root  Mean  Square  (Root  MSE) 


1294.163 

24 

53.9234 

7.3432 


4.  Estimate  of  b-Value 

Variable  DF  b-Value 

(Intercept)  0  0 

V  1  3.7833 


Std.  Deviation  t  Ratio  Approx. Prob. 

0 

0.4537  8.337  0.0001 


5.  Covariance  of  b-Value 


Intercept 


Intercept 


V 


t 


0  0 
0  0.2059 
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Table  C-2 (Continued) 


Estimates  of  Lag  Coefficients 

Coefficient  Std.  deviation  t  ratio 

-0.2833  0.1890  -1.4984 

-0.1093  0.1890  -0.5784 


Sum  of  Squares  (SSE) 

Deg.  of  Freedom  (DFE) 

Mean  Square  (MSE) 

Root  Mean  Square  (Root  MSE) 

Estimate  of  b-Value 
Variable  DF 

(Intercept)  0 
Vt  I 

Covariance  of  b-Values 

Intercept  V 

Intercept  0 


b-Value  Std.  Deviation  t-ratio  Approx.  Prob 
0  0 

2.5815  0.2754  9.371  0.0001 


861.9477 

24 

35.9144 

5.9928 


Lag 

1 

3 

Error  Terms 


V 


0 


0.0758 


C.2  Time  Series  Plots  of  the  Data 


The  data  for  each  pre-strike  and  post-strike  period  did  not  show  a 
significant  relationship.  Therefore,  it  was  decided  to  generate  a  time 
series  plot  of  the  overall  data  and  to  further  investigate  for  any  possible 
clues  or  relationship  which  may  lead  to  a  meaningful  relationship.  The  series 
plots  are  shown  in  Figures  C-l  and  C-2. 

Additional  analysis  on  autoregression  was  performed  and  presented  as  a 
comparison  of  pre-strike  and  overall  data  in  Figures  C-3  and  G-4.  The  plots 
show  that  most  of  the  observations  fall  within  the  confidence  regions. 
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Terminals 


Figure  C-l.  Time  Series  Plot  of  Error  Rates  in  Terminals 


Time  Series  Plot  of  Error  Rates 
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Figure  C-3.  Relationship  Between  Tower  Operational  Errors  and  Number 

of  Operations  Handled  (  Overall  Pre-  and  Post- Strike  Periods) 
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Figure  C-4.  Relationship  Between  Center  Operational  Errors  and  Number 

of  Operations  Handled  (Overall  Pre-  and  Post-Strike  Periods') 


I.  Comparison  of  Pre-Strike  and  Overall  Data 

The  comparison  between  the  pre-strike  and  the  overall  data  reveals  some 
dramatic  differences.  The  overall  data  indicate  improvement  in  the  correla¬ 
tion  coefficient.  In  the  towers,  R2  =  0.5779,  and  for  the  centers,  R2  *  0.4474. 
Both  are  significant.  The  sigma  value  was  smaller  in  the  centers  data  than  in 
the  towers  data.  The  plots  of  the  two  along  with  confidence  regions  are  shown 
in  Figures  C-5  and  C-6. 

In  comparing  the  autoregression  model  of  both  pre-  and  post-strike  data, 
it  can  be  noted  that  the  autocorrelations  are  all  positive  in  all  three  of  the 
time  lags  in  the  overall  data.  In  the  pre-strike  data,  the  autocorrelations 
are  negative  in  the  first  and  second  order  time  lags  and  change  sign  in  the 
third  order.  When  the  autoregressi\e  estimates  were  entered  into  the 
derivative-free  nonlinear  least-squares  estimation  procedure,  it  was  found 
that  the  residual  sum  of  squares  converged  after  the  third  iteration,  in 
both  pre-strike  and  overall  data. 

The  sum  of  the  squares  for  the  regression  parameter  is  lower  in  overall 
data  than  the  pre-strike  data.  Also  in  the  pre-strike  data,  the  confidence 
interval  is  wider  than  the  overall  data.  And  all  the  residuals  post-data 
points  in  the  overall  data  are  negative,  thus  showing  that  the  errors  are 
lower  thanwould  be  expected  by  the  model.  Since  the  model  has  a  better  fit  in 
the  overall  data  than  the  pre-strike  data,  it  is  reasonable  to  deduce  that  the 
system  is  at  least  as  safe  as  in  the  pre-strike  period. 

The  same  is  true  for  the  centers.  In  the  pre-strike  data  the  autocorre¬ 
lations  were  very  low.  The  estimates  of  B  value  for  the  independent  variable 
were  significant  at  0.0001  level  in  both  pre-strike  and  overall  data.  The 
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Relationship  Between  Tower  Operational  Errors  and  Number 
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Figure  C-6.  Relationship  Between  Center  Operational  Errors  and  Number 

of  Operations  Handled  (Overall  Pre-  and  Post-Strike  Periods) 


range  of  confidence  interval  was  wider  in  the  pre-strike  data  as  shown  before 
in  the  tower  data.  The  residuals  were  again  negative  in  the  overall  data. 

It  would  be  reasonable  to  deduce,  then,  that  the  system  is  at  least  as  safe 
in  the  post-strike  period  as  in  the  pre-strike  period.  Figure  C-7  shows  typi¬ 
cal  statistical  tests  performed  by  Statistical  Analysis  System  (SAS)  computer 
programs. 
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Figure  C-7.  Typical  Statistical  Tests  (Continued) 


APPENDIX  D.  COEFFICIENT  OF  CORRELATION  AND  TEST  OF  PROPORTION 


D. I  Coefficient  of  Correlation 

The  method  used  was  a  Pearson  product-moment  correlation  coefficient.  The 
formula  used  is  as  follows: 


where  =  volume  of  operations 
=  operational  errors 


D-  2  Test  of  Proportions 

This  method  uses  Fisher's  z  distribution.  The  formula  used  is  as  follows: 


pb  -  pa 


pq  (  —  +  ~“) 

%  na 


where  =  proportion  or  error  rate  in  the  pre-strike  period 
p  *  proportion  or  error  rate  in  the  post-strike  period. 
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APPENDIX  E.  PLOTS  OF  OPERATIONAL  ERRORS  BY  WEEK 


Figures  E-l  and  E-2  show  the  time  series  plots  of  operational  errors  by 
week  in  the  terminals  and  centers.  The  difference  between  the  pre-strike  and 
post-strike  operational  errors  in  the  terminals  is  larger  than  corresponding  error 
in  the  center.  When  the  operational  errors  are  combined  in  the  terminals  and 
centers,  the  plots  show  that  the  pattern  of  errors  is  similar  in  the  post¬ 
strike  period  to  that  of  the  pre-strike  period.  Post-strike  errors  show  lower 
level  than  pre-strike  period  errors.  However,  when  the  data  are  plotted  using 
the  methods  of  moving  average  with  a  step  of  three,  the  pre-strike  errors  show 
smoother  level  than  the  post-strike  errors.  These  plots  are  shown  in  Figures 
E-3  and  E-4  respectively. 

The  operational  error  rates  for  the  terminals  and  centers  are  shown  in 
Figures  E-5  and  E-6  by  week.  It  is  evident  that  the  post-strike  error  rates 
are  lower  than  the  pre-strike  error  rates.  The  corresponding  mean  level  is 
lower  in  the  post-strike  period  than  in  the  pre-strike  period.  The  two  points 
which  are  circled  are  asymptotic  points.  In  the  center  the  data  range  is  wider 
than  that  of  the  terminals. 


m  oo|- 


m  r-|co 


O 

°2 


5  'Ol'O 
3  mi£ 

CM 

»r  in  |  O' 

o  ..in 


ao  ^ 


£  _.,ao 
cn  r>\^ 

3 


fN  ,ao 
m  ^  I  cm 

m  cmI^h 
rM  Hln 

>0  r-i  |  ^ 
"M 

•3 

3  2IS 
3  2f* 

s  -la 


olm 
I  .-I  I  CM 


•  Ol»H 
I  -tli-t 

!  <*i£ 


I  Ox|2  CO 

'•IS  I 

u 

ooi2  « 

01 

0J 

I  00  CM  3 


<0 


4-i 

O 


O 

eu 

CO 

<U 

*H 

0) 

CO 


CO 

I 

w 

<u 

3 

00 


Operational  Errors  by  Wee 


Figure  E-5.  Time  Series  Plot  of  Operational  Error 


